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Preface 


There is much to discuss, and debate, about the sources and effects of inequality on economic 
growth and social mobility, and the relationship between education and economic growth. 
However, an undergraduate degree from a highly selective and elite college gives students a 
strong advantage in gaining employment in jobs that promise greater lifetime earnings. The 
widely held assumption is that the prestige of the college is a useful proxy for the quality of the 
graduates of that institution. Irrespective of the relationship between education and economic 
growth, post-secondary education is imperative to the enhancement of human capital 

and arich source of talent for employers. If there are significant bottlenecks, or structural 
impediments, that block equal opportunity for students of high ability that do not go to elite 
colleges, we should identify the problem and attempt to reduce or eliminate it. 


In the United States, achieving equal opportunity in post-secondary education is typically 
described in terms of enrolling more underrepresented groups in selective colleges. The belief is 
that if this step is accomplished, it will have a fundamental impact on the problem of inequality 
at the national level. However, what if there are not enough places for students in selective 
colleges to accomplish this goal? What if selective colleges do not have enough capacity to 
make a significant impact on the problem of serving students from underrepresented groups 
with demonstrated high abilities? The rest of this paper will address these questions. 


Introduction 
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The post-secondary education sector is beset by a number of severe headwinds. Cost issues 
now elicit comparisons with the healthcare sector’s cost problem. Access deficits continue 
to rise, and retention and graduation rates are unacceptable. Questions about the quality 

of education are increasing as well. Moreover, the rise of online and competency-based 
undergraduate programs, emerging across the for-profit, non-profit, and public sectors, 
constitutes a disruptive force that threatens the business model of the traditional brick-and- 
mortar, post-secondary education sector. 


However, it is important to remember that higher education was formed to achieve two 
principal objectives: 1) to provide education for students, encouraging them to achieve their 
highest intellectual potential, and 2) to encourage faculty to teach and produce scholarship 
and research to their highest level of ability. A principal role, introduced by Thomas Jefferson, 

is for higher education to be the major source of social mobility for all citizens. This ideal was 
institutionalized as the Morrill Land Grant Act of 1862. It is this principle that | focus on here. 
Today the college that students attend largely determines their economic and social chances in 
life. The question is whether the level of prestige granted to a select group of institutions unduly 
restricts the ability of all students to benefit equally from the educational attainments they 
demonstrate. Do all graduating college seniors secure jobs commensurate with their skills? 

If the answer is no, can we identify innovations that assist in leveling the playing field for all 
citizens? 


Equal Opportunity Not Equal Results 


.. if Smith and Jones have the same native talent and Smith is born of wealthy, 
educated parents of a socially favored ethnicity and Jones is born of poor, 
uneducated parents of a socially disfavored ethnicity, then if they develop the 
same ambition to become scientists or Wall Street lawyers, they will have the same 
prospects of become scientists or Wall Street lawyers if equal fair opportunity 
prevails. (Rawls, 2001, “A Theory of Justice,” section 13) 
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This statement captures how the concept of equal opportunity is used here. 


Since the premise | am beginning with is that there is not a level playing field for all graduating 
college seniors, a basic question is: Why have we not attempted to create and implement 
innovations to deal with the problem until now? Here is a good answer: 


Goldman Sachs doesn’t intrinsically care about Harvard. They care about finding 

the best person for the job. Elite brand degrees have just traditionally been the best 
proxy metrics for that because precise metrics weren’t heretofore available. (Ferreira, 
2013, “Disruptive Innovation vs. Harvard”) 


We now have precise metrics that can be used to offset the power of the proxy of elite brand 
institutions. The rest of this paper suggests how we can benefit from using them. | will use the 
Collegiate Learning Assessment (CLA), in particular the recent next generation of this protocol, 
CLA+, for my illustrative argument because | am most familiar with this assessment.” 


The Context for the Problem 


If the absence of equal opportunity is the problem, can higher education be altered to create 

a more level playing field for all graduating college seniors? And, what is the premise behind 
this question? My argument is that, as in most countries, a set of selective colleges can be 
described as a positional good that determines the life chances of most students. The concept 
of positional good (Hirsch, 1976) describes goods that fall under the category of “zero sum,” 
which means there are upper limits to their consumption. Only a finite number of people can 
attend a chamber music concert before it ceases to be one. The same thing is true about many 
other goods in life, including top positions in the private and public sector. The places at highly 
selective colleges are limited; again by definition, a selective college would cease to be one if 
it grew beyond perceived enrollment limits. (This itself is a contentious point for some.) Here | 
will attempt to provide empirical evidence that selective colleges do not enroll the vast majority 
of high-ability students (this term defined below). If this hypothesis is borne out, we have a 
maldistribution of our human capital at the societal level and an unequal playing field in the 
college-to-career space. 


Second, are significant innovations in major social institutions, such as higher education, 
possible? The institution of higher education itself is rightly thought to be one of the major 
social institutions in American society, highly institutionalized and thus not easily transformed 
by outside disruptive forces. However, it is important to remember that humans create 
institutions and not the other way around (Harsanyi, 1969). Since the education system is the 
only venue we have to preserve and enhance human capital, it is prudent to audit it from time to 
time to judge whether it is in need of redesign. This is particularly true for U.S. post-secondary 
institutions because they have greater importance in today’s Knowledge Economy, where 
national economies that strive to stay at the forefront of the new product-innovation cycle and 
the ideas that generate the highest economic value added are the winners. 


Finally, does educational technology provide new ways to innovate in post-secondary 
education? The reason this question is important is because education technology now 
provides a cornucopia of potential innovations for higher-education institutions. 


There are now technology-based solutions, including open-education resources, flipping 
classrooms, and adaptive and personalized instruction. Advances in education-assessment 
tools, the reason for the innovation suggested here, has been stimulated by recent investments 
of over $360 million by the U.S. Department of Education in twenty-first-century tests, in 
support of the Common Core movement in K-12 education. As a byproduct, new ways to use 
educational technology have led to novel assessments—such as interactive games—that give 
promise of being widely implemented. 


Significant innovation, in support of institutional redesign of important segments of higher- 
education institutions, is possible. We appear to be at a moment when we can think practically 
about re-engineering key processes to make the higher-education sector more efficient and 
more effective. 


1 See also Fishkin (2014) for a new treatment of the implications of equal opportunity. 

? The other two national assessments are Proficiency Profile (Educational Testing Service) and The Collegiate Assessment of Academic Progress 
(ACT). These three critical-thinking tests were found to be reliable and valid in a test validity study sponsored by the Department of Education, 
Fund for the Improvement of Postsecondary Education. See Klein et al. (2009) cf Steedle, et. al. (2010). 
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The Case for a Market Failure 


Today, going to college is the principal means to success in the United States. However, all 
post-secondary education institutions are not equal; a few are viewed as selective colleges. 
These colleges, with the lvy Leagues at the apex, are examples of positional goods. Only a 

small percentage of college students enter and graduate from these colleges. Many leading 
companies recruit only from this group of colleges. The most selective colleges and universities 
have strong barriers to entry. Students who win enrollment to these colleges tend to have the 
advantage of significant financial and social support from early childhood through high school. 


Parents, with the appropriate financial means, are willing to financially support their children 
in gaining the skills needed for admission to Yale because that gives their children entry into 
the select circle of society’s “winners,” economically and socially. They have the resources to 
compete for top public and private leadership positions which, by definition, are finite, in short 
supply, and possess a zero-sum quality. There, typically, is only one CEO of a company, one dean 
and one president, respectively, of a university, and so on. Of course, selective colleges have 
scholarships and affirmative action policies that permit them to enroll minority students, but 
the resources devoted to these policies are not enough to have a significant impact on creating 
a truly diverse student body. 


The gap between the per-student endowment of Yale versus a public university, such as the 
City University of New York, is so large that it is difficult not to conclude that education at Yale 
is avery different student experience, which gives graduates huge life-long advantages in 

their individual human capital assets. No one should want to harm Yale’s ability to deliver an 
education of the highest quality. However, as the support for public higher education wanes 
across the United States, we must be prepared for even greater economic and social inequality, 
less social mobility, less diversity, and, in all likelihood, less economic growth. Why? Because 
the selective colleges may not supply a sufficient critical mass of educated citizens to maintain 
the U.S. human capital comparative advantage globally. Surely we should examine what might 
be done about this issue. 


The rich diversity of American post-secondary education is correctly cited as a unique 

strength. However, just as we made significant changes in admissions requirements for college 
applicants in the aftermath of World War II by creating admissions tests, such as the SAT and 
ACT, to complement the high school GPA, we now face the need to create a test to complement 
the graduating college senior’s GPA. We should do so to reset the opportunity structure held in 
place by the positional-good world of post-secondary education. However, we should not attack 
selective colleges but design a way to widen the opportunity structure for students in less- 
selective colleges. Why should we do this now? 


The positional-good system of selective colleges may have made sense in an earlier age of 
American development. But with a population of over 320 million (to be 400 million in three or 
four decades), the bulk of whom live well outside the geographic reach of most of the selective 
colleges, we need to dramatically expand the opportunity system created by post-secondary 
education. This is especially true for underrepresented high-ability students because they 
appear to attend less-selective colleges in overwhelming numbers. What follows is an attempt 
to justify this problem statement and suggest a way to attack it. 


The Problem to Solve 


Hoxby and Avery (2012) demonstrate that as many as 10 to 15 times the number of African 
American and Hispanic students, as previously thought, have SAT or ACT scores that meet 

the admissions requirements of the most prestigious colleges in the United States. However, 
these students are often advised not to apply to selective schools and attend high schools not 
visited by college recruiters from elite institutions. If we did not have the SAT and ACT in place, 
admissions officers would only have students’ high school GPAs to rely upon for admissions 
decisions. There likely would be even fewer students from underrepresented groups admitted to 
the most selective colleges because the SAT and ACT provides important additional information 
to students’ high school GPAs (Kobrin et al., 2008). 


3 The per student endowment at Yale University is $1,750,000 while the per student endowment at the City University of New York ranges from a 
few hundred dollars at Medgar Evers College to $9,000 at Baruch College. Voluntary Support of Education, VSE Data Miner, cae.org. 
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College to work presents a more severe market failure because there are no tests to accompany 
students’ college cumulative GPAs that could control for the grade inflation and variability of 
grades across colleges. Such a test could assist in leveling the playing field for students from 
less-selective colleges without damaging the prospects from selective colleges. 


Students 


Grade inflation has resulted in the national mean college cumulative senior GPA rising to 3.3 
(on a four point scale) (Rojstaczer and Healy, 2012). This means that most graduating seniors 
do not have an objective way of distinguishing their skills from other students when they apply 
for jobs; they are all above average. Students who attend the top 150 selective institutions 

are likely to get a pass because many employers will choose students based on institutional 
prestige. However, what about the others, the 90-plus percent of graduating seniors who attend 
less selective institutions? Moreover, what about the most disadvantaged students? There are 
a large number of low income students who graduate from these colleges and universities that 
have the critical-thinking skills and abilities that employers cherish. 


Colleges 


Less-selective colleges produce many college graduates who achieve distinction in their 
careers. However, they face a branding problem. Since these colleges do not have reliable 
tools that make the case for their stronger graduating seniors, employers may never discover 
their students. Less-selective colleges, in particular, should consider recommending that their 
graduating seniors take CLA+. This should increase the number of their graduates at the peak 
of the value-added economic product cycle. If this turns out to be the case, less-selective 
colleges will change employers’ preconceived notions about their graduates and the colleges 
that produce them. 


In sum, the market failure between graduating college seniors in less-selective colleges seeking 
employment and employers is blocking hundreds of thousands of students from attaining 
employment appropriate for the high-ability skills they have attained. Too many students do 

not get to interview for jobs that they have the skills for because employers are unaware of 
them. This is bad for the students in question, their institutions, and employers. At the macro 
level, it means too much of our human capital is mal distributed. And it also is a major block to 
opportunity for high-ability students from disadvantaged backgrounds who attain skill levels 
equivalent to their selective college counterparts. 


Employers 


Employers spend much time and money interviewing potential applicants for jobs. Some 
employers give applicants assessments on the skills and abilities they require in their 
employees. However, in the age of grade inflation, how do hiring managers decide who to 
interview in the first place? If, in addition to résumés and college transcripts, hiring managers 
have the results of a valid and reliable critical-thinking test, their pool of potential applicants 
would be enlarged. The employment process would be more effective and efficient, and 
employers would be better equipped to tap the social, economic, and ethnic diversity of 
students reflected in all of our colleges and universities. 


Increasingly, employers have been forced to ask applicants to send their SAT/ACT scores 

or Graduate Management Admission Test (GMAT) scores along with their resumé because 

the high level of grade inflation means the transcripts of graduating seniors do not supply 
sufficient information about the job candidates’ skills. This is unacceptable. The GMAT, a solid 
critical-thinking test, is targeted for a small population of students applying for business- 
school programs. The SAT is not an acceptable measure for the skill levels college seniors 
have achieved, for two reasons. First, reliance on a test that applicants took before they 
entered college means no credit is given to the effect of the college experience. This not only 
unacceptable, it is not credible. We know from analysis of the CLA database that the overall 
mean student-learning growth, for college students from freshmen to seniors, is .73 (a .44 
standard deviation). This finding is based on analysis of all students taking the CLA in over 1200 
test administrations over the past eight years. This is a significant effect. College does matter 
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to student-learning growth, and it matters a lot (See Benjamin, 2014). We should not assume 
that college does not matter. Moreover, we should not ignore the strong probability that many 
of these same students and their Hispanic and non-Hispanic white counterparts who attend 
less-selective colleges exhibit above average growth in their critical-thinking skills as a result 
of their college experience, even if they do not attend highly-selective colleges. 


The Challenge 


If employers had the results of a reliable and valid pre-screening assessment of graduating 
college seniors, that might provide suitable information to enlarge the pool of potential 
applicants. If less-selective colleges produce many college graduates that achieve high levels 
of the skills employers cherish and can overcome the problem of employers not knowing how 
to find these students, these institutions will change the perception of employers about their 
graduates and the colleges that produce them. 


The hypothesis, then, is that less-selective colleges graduate a large number of high-ability 
students (as defined by appropriate standardized assessments), as many or more students 
than the selective colleges that serve as the proxy for excellence as noted by Ferreira on page 
two. 


Validating the Market Failure Thesis 


There is at least one data set that may be useful for this purpose, the data set of the Collegiate 
Learning Assessment (CLA and CLA+). There are two other reliable assessments in the 
graduating senior space, the ETS’s Proficiency Profile and ACT’s Collegiate Assessment of 
Academic Progress (CAAP), that are also potentially appropriate for this purpose. What is CLA+? 
Is it reliable and valid? 


CLA+ measures critical-thinking skills, regarded as the top priority by employers (Hart 
Associates, 2013), including: 

* analysis and problem solving 

¢ writing mechanics 

¢ writing effectiveness 

* scientific and quantitative reasoning 

* critical reading and evaluation 

* critique an argument 


These cognitive skills are independent from academic disciplines and are teachable. They are 
thought to be particularly important skills in today’s Knowledge Economy where one can Google 
for facts and, hence, the question becomes whether the student can access, structure, and use 
information, not just whether he or she can remember the content; these skills are highly prized 
by faculty and college leaders who, like their K-12 counterparts, are moving toward “deeper” 
learning in both K-12 and higher education (Benjamin, 2014). * 


CLA, and the updated CLA+, has been used in over 700 colleges and universities in the United 
States, many testing on an annual basis since 2004-05. It has been used internationally in 
approximately 125 colleges in 12 different countries. 


* For evidence of reliability and validity see the following sources. Klein and R. Benjamin. The CLA+ at cae.org. See also the 90 studies of 
reliability and validity listed and analyzed in R. Benjamin et al. (2013), The Case for Critical Thinking Skills and Performance Assessment at cae. 
org. For important new validity evidence see R. Arum and J. Roksa, forthcoming 2014, which shows that graduating seniors who do well on CLA+ 
are in better financial situations and are much more likely to be employed than those students who do poorly on CLA+. 
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Figure 1 Distribution of Senior CLA Scores 
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Figure 1 indicates that 68% of the students fall within 1 standard deviation of the mean score 1159. The question is how 
many of the students scoring in the top 10%, 25%, and 50% are in selective versus less-selective colleges (see Appendix C). 


Table 1 
Proportion of Selective vs. Less-selective Institutions 


Colleges & Universities Student Enrollment 
1980 2012 1980 2012 


Selective 143 (5%) 143 (5%) 762,248 (12%) 940,771 (9%) 
Less Selective 3,014 (95%) 3,014 (95%) 5,584,841 (88%) 9,823,718 (91%) 
ALL 3,157 (100%) 3,157 (100%) | 6,347,089 (100%) 10,764,489 (100%) 


Table 1 was constructed to set the context for our test of whether a market failure exists. Over 
the past 30-plus years, the number of students in the 143 selective colleges has grown by 
171,000. Over that same time period, the number of students attending less-selective colleges 
has grown by over 4,200,000. The largest growth in four-year college attendance is in the less- 
selective colleges. 


The list of selective colleges is based on the Barron’s selectivity college index (See appendix C 
and D), the same index used by Hoxby and Avery (2012). Of course, any division of colleges and 
universities into selective and less-selective categories can be challenged. For example, today 
Indiana University, the University of California, San Diego, and private liberal arts colleges, 
such as Earlham College and Kalamazoo College, would appear to warrant the label selective. 
The Barron’s selectivity index reflects judgments based on past performances which may not 
capture recent trends. Based on the analyses here, the list of selective colleges itself should 
be widened. See Appendix C for information on the data sources. Note that the numbers 
above for the Colleges and Universities column do not necessarily reflect all 4-year degree 
granting colleges and universities in existence in 1980 and 2012; these numbers come from 
an enrollment data set from the Integrated Postsecondary Education Data System (IPEDS), so 
there may be institutions not included here simply because they did not provide enrollment 
data to IPEDS. 
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Table 2 
Projected National CLA Performance 


A Exiting Seniors at CLA Institutions 


Actual Selective Less-selective 
CLA Performance Institutions Institutions 


Above 1400* 395 (24%) 2,631 (6%) 3,026 (7%) 


Above 1300 841 (52%) 8,001 (18%) 8,842 (20%) 
Above 1200 1,284 (79%) 17,956 (41%) 19,240 (43%) 


ALL 1,627 (100%) 43,352 (100%) 44,979 (100%) 
* These scale points are based on the CLA scale, which like the “old” and upcoming SAI, ranges from 400 to 1600. 


Bachelor’s Degree Recipients Nationally (2011-12) 


B and Projected CLA Performance 


Projected National : : 
CLA Performance Selective Less-selective ALL 


Institutions Institutions 


Above 1400 53,307 (24%) 95,295 (6%) 148,602 (8%) 
Above 1300 113,497 (52%) 289,796 (18%) 403,293 (23%) 


Above 1200 173,282 (79%) 650,365 (41%) 823,648 (46%) 
ALL 219,572 (100%) 1,570,207 (100%) 1,789,779 (100%) 


*Note that the total national percentages differ somewhat from the percentages of students at all CLA institutions scoring 
at given levels, due to a slight underrepresentation of selective colleges taking the CLA. 


Using the percentages of students above 1400, above 1300, and above 1200, Table 2 shows 
that selective colleges produce a higher percentage (24%) of students above 1400 than the 
less-selective colleges (6%). However, there are almost twice as many high-ability students 
graduating from less-selective colleges above the 1400 level. The proportion of high-ability 
students in the less-selective colleges grows for students testing above 1300 and 1200 (see 
graph). The less-selective colleges have large percentages of low-income (Pell grant) and 
moderate-income students from diverse backgrounds. 


Graph 1 captures this finding. A significant market failure exists in the college-to-career space. 


Graph 1 
Projected National CLA Performance 
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Geographic Distribution of Selective and Less-selective Colleges 
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The map shows that the selective colleges are isolated from most students in the United 
States. The 143 selective colleges, largely in the northeast, form a positional-good gatekeeper 
system that overly determines the life chances of all students across the country. We know 
most students attend college within commuting distance. The location of the selective colleges 
means there is a disconnect with large and growing ethnic/racial and income groups throughout 
the United States. 


Table 3 
Distribution of Student Race and Ethnicity, by Institutional Selectivity, 1980 and 2012 
Students 1980 2012 


ALL 1980 ALL 2012 Selective Less Selective Selective Less Selective 
Institutions Institutions Institutions Institutions 


Non-Hispanic White 


Hispanic 


Black or African American 
Asian or Pacific Islander 
Other 


Table 3 provides further evidence of the market-failure challenge. First, there has beena 
significant change in distribution of race and ethnicity by institutional selectivity from 1980 to 
2012. The selective colleges are more diverse. The main point, however, is that the increasingly 
small proportion of total college enrollment made up by selective colleges reinforces the 
market-failure thesis. The major growth of enrollment is in the less-selective college group. 


Possible Solution 

aa ee 

The goal is to develop tools that allow for the demonstration of skills that are important for both 
employers and students and to make that information accessible to students and employers. 
CAE has developed one process, labeled “CLA+ as a Work Readiness Pre-Screening Tool in the 
College-to-Career Space” (see Appendix A). It is being piloted to see if it can reduce the market 
failure. 


Each student receives a score report that indicates the level of mastery for skills measured 

by CLA+. Qualifying students (those with proficient- or advanced-mastery levels) may claim 

a certified badge through a secure vault hosted by Pro Exam. Students will be able to store 
their score reports with online transcript service providers and place their CLA+ scores on 
employment boards. Qualifying students (those with top scores) were invited to a virtual career 
fair, hosted by Brazen Careerist, in May 2014 to meet with selected employers. Employers 

who attended the virtual career fair did so knowing that the students they met possess the 
mastery level critical-thinking skills they regard as requisites for an interview. Moreover, they 
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knew that they met many students who have the diverse backgrounds they are seeking in order 
to diversify their workforce. Both employers and students saved time and money through the 
virtual career fair meeting. 


Figure 2 illustrates the CLA+ Career Connect system. 
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The steps outlined above are possible for graduating seniors who take CLA+ and elect to add 
the additional information to their transcript, secure a certified badge, if eligible, and send 
their results to respected jobs boards accessed by employers. If students qualify, they can also 
attend a virtual career fair all of which will improve the odds of high-ability students in less- 
selective colleges to obtain a good job and start a promising career. 


Benefits of a 
More Level Playing Field 


When groups do not believe the playing field is level, they become discouraged and drop out. If 
enough students and enough colleges and enough employers embrace the innovation outlined 
here, then the market failure will be reduced. Most economists agree that an open, transparent 
market brings greater benefits to buyers and sellers. Ways to reduce the “noise” between 
college and career should thus be encouraged. This would result in a better distribution of 
human capital and a more level playing field. 


Conclusion 


Olson’s collective action model (1965) suggests that new innovations are needed to help create 
a more level playing field. We cannot expect stakeholders at the selective colleges to deal 

with the problem alone. New innovations are needed to reset the rules of the game students, 
employers, and colleges operate within. There is a strong case for use of an appropriate 
standardized assessment to provide graduating college seniors and employers additional 
information about their critical-thinking skills. Such an assessment could be used as a sourcing 
tool by employers which would be of assistance to them and the students they identify assisted 
by the measurement tool. 


The “tunnel effect,” formulated by economists A. Hirschman and M. Rothschild, suggests why a 
more level playing field for careers encourages members of all ethnic/racial groups to compete/ 
work hard. Imagine you are in the left-hand line of two lanes of traffic held up in the Lincoln 
Tunnel. Frustration builds. Finally, cars in the right-hand lane begin to move. What are your 
emotions then? Hirschman posits that you are excited and now positive about the possibility 
that your lane of traffic is next to move. You see light at the end of the tunnel. However, what 

are your reactions when your lane does not move forward? You realize that while cars in lane 2 
moved forward, your lane is stuck. You are stuck. That is when feelings of anger or despair set in. 
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The United States is only one example of the market failure. Based on countries that | have lived 
in, many countries probably exhibit significant disconnects between their higher-education 
systems and employment. Because many countries may face similar market failures between 
college and career, as outlined in this case study of the U. S., research programs should be 
formed to refute or corroborate the points | have made and create new evidence-based findings 
on the topic across nations. 


Most importantly, the principle of equal opportunity is widely shared in liberal democracies 
(Hartz, 1955). “Equality has always been the most radical and potent idea in American history 
(Wood, 2014, p. 38). If it is problematic in the senior-to-career space for many students to find 
jobs appropriate to their skills, then that is a serious problem that we need to fix. Today, we have 
metrics to measure the quality of the skills that colleges and employers indicate they prize 
highly. 


We also have education technology-based tools to more quickly inform employers and students 
about the cognitive skills required for jobs and the skills that students have that appear to meet 
those requirements. 


The market failure problem between graduating college seniors and employers should be very 
high on the policy agenda to solve by both private and public stakeholders involved. It is a 
matter of significant national interest to more efficiently allocate the nation’s human capital, 
the only resource any nation has, and to improve equal opportunity, which is key to improving 
economic and social equality. 
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Appendix A 


The Case for CLA+ Career Connect as a 
Work Readiness Pre-Screening Tool in the College-to-Career Space 


CLA+ CareerConnect has three distinct but interrelated purposes for the three principal 
stakeholders in the college-to-career space: employers, students, and colleges. All three goals 
are intended to reduce the market failure that has resulted in a disconnect between colleges 
and employers and between graduating college students and employers. The first purpose is 

to provide a pre-screening tool for employers. The second purpose is to validate, augment, and 
frame the growing number of credentials, organized as digital badges, for students. Colleges 
and their instructors may also use CLA+ to validate the badges they produce. The third purpose 
is to validate the crucial role colleges have in producing students with the proficiency in critical- 
thinking skills that professors and employers agree are requisites for success in the workplace. 


The main premise of utilizing CLA+ CareerConnect as a prescreening tool is that additional 
credible information about the overall critical-thinking skills of students is needed to 
accompany the information provided by college transcripts. Only a standardized test that 
measures critical-thinking skills, such as CLA+, which is widely seen by the measurement 
science community as reliable and valid, is appropriate for this purpose. It is crucial for 
tests that are built by measurement scientists to be reliable and valid in scientific terms 

to accompany the rich granular information provided by digital badges and workplace 
assessments. 


1) CLA+ as a Pre-Screening Tool for Employers 
Types of Tests for Employers 


A large amount of time and energy has been spent designing assessments employers can use 
to make hiring decisions. Therefore, most of the attention devoted to assessments for work 
readiness is focused on the human resources department. Although assessment protocols 
for human resource departments may be tailored to meet a specific employer’s needs, the 
assessment categories may be divided as follows: 


¢ Aptitude tests—predictive of job success in specific fields and focus on abilities, such 
as spatial visualization, verbal conceptualization, and math aptitude. These tests are 
expensive, time consuming, and must be updated frequently to remain aligned with 
changing job requirements. 

¢ Personality tests—“..instruments for the measurement of emotional, motivational, in- 
terpersonal and attitudinal characteristics as distinguished from abilities...” (Anastasi 
& Urbina, 1997). For example, characteristics such as one’s openness to experience, 
extroversion, agreeableness, conscientiousness, emotional stability, well-being, and/or 
happiness are measured. 

e Academic discipline tests—tests of content knowledge for particular fields and 
domains. For example, these tests help answer questions like: Does an applicant fora 
position in accounting have the requisite content knowledge to do the job? 

* Cognitive abilities tests—assess critical-thinking or cognitive skills (see below), but 
these tests are also extended to soft skills such as persistence, collaboration, entre- 
preneurship, creativity, collaboration, and moral or ethical reasoning (Kraiger et. al., 
1993). 


The most prominent example of a work readiness assessment is ACT’s WorkKeys (2014), which 
comprises a combination of these types of tests. This test, crafted to predict job success 

in thousands of occupations, most of which are in the sub-baccalaureate market, requires 
applicants to demonstrate familiarity with the skills needed for success in the occupation, but 
the test also predicts the ability of applicants to succeed. 


The Criteria for the Evaluation of Tests 


No matter what combination of test items make up an assessment protocol, reliability and 
validity, along with time and cost, are always the basic criteria used to evaluate the test. 
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Validity concerns the extent to which the test measures the knowledge, skills, and abilities it is 
designed to measure. Reliability refers to the degree of consistency of students’ scores across 
a test’s questions, the consistency of scores across different assessors, and whether the tests 
are given to students under the same conditions and over the same time period and, thus, meet 
the essential criterion of fairness (Klein, 2002). The measurement science community views 
these two criteria as necessary for an effective assessment. 


Time and cost are also crucial. Assessments that require more than two hours are restricted 
to high-stakes tests, such as the SAT, MCAT (medical school), and LSAT (law school). Tests that 
cost more than $75 to $100 are products for niche markets; for example, certification tests for 
specialized professional fields such as piloting an airplane. 


The types of tests listed above vary on the criteria noted. Aptitude tests are costly and time- 
consuming to administer. Academic discipline tests, geared towards specific fields, are 
possible to administer, but it is not easy to compare potential applicants across diverse fields. 
Personality tests are used widely but are not considered highly reliable by the measurement 
science community (Stabile, 2002). 


Cognitive abilities tests are not only considered a significant requisite for employment success, 
they are the only tests with sufficiently high levels of reliability and validity (as defined by 
measurement scientists) that permit wide comparisons across all graduating college seniors, 
and are also efficient in testing time and cost. Cognitive abilities can also be defined as critical- 
thinking skills, which are regarded as the top priority by employers (Hart Associates, 2013) and 
include: 


¢ Analysis and Problem Solving 

¢ Writing Mechanics 

¢ Writing Effectiveness 

¢ Scientific and Quantitative Reasoning 
¢ Critical Reading and Evaluation 

¢ Critique an Argument 


Faculty surveys also place these cognitive abilities at the top of requirements for graduating 
college seniors (Hart Associates, 2013). Moreover, most colleges feature these skills in their 
mission statements or general education goals. 


Three assessment organizations, ETS, ACT, and CAE, cooperated in a test validity study of 

the two multiple-choice tests that assess these critical-thinking skills developed by ETS 

and ACT, respectively, and the performance-based assessment developed by CAE. All were 
judged reliable and valid by the measurement scientists at these three education assessment 
organizations. However, this does not mean, necessarily, that all of the assessments measure 
the same criteria (Steedle, et al., 2012). 


Many have suggested that measures of soft skills, such as entrepreneurship, collaboration, 
creativity, and moral or ethical reasoning, should be added to the above list of prioritized 
cognitive skills. These additional measures have not yet received the endorsement of the 
measurement science community in regards to reliability and validity (see Hersh, et al., 2012). 
Therefore, adding the soft cognitive skills to the above listed cognitive skills must wait for the 
research and development process to prove their reliability and validity. Therefore, development 
and implementation of combinations of the above types of tests for use by human resources 
departments to make hiring decisions will remain a long-term, perhaps ongoing, set of issues to 
solve for. 


The Case for a Pre-Screening Assessment 


Given the numerous combinations of work-readiness assessments and the difficulties 

involved in designing assessment protocols that meet the minimum conditions required by the 
four criteria of reliability, validity, time, and cost, it is understandable that a healthy debate 

is underway to clarify which strategy, or strategies, make the most sense for developing 
assessments that meet the needs of human resource departments. However, the implication of 
the market failure argument presented here suggests the emphasis on assessments for human 
resource departments is not the most urgent problem to focus on. 


Employers indicate there is a significant skills gap in the United States. There are an estimated 
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10 to 20 million vacant jobs in the US (Economic Modeling Specialists International, 2014). The 
number of places for students in selective colleges remains static at under one million (see 
Baron’s selectivity index). Employers need methods to distinguish the large number of students 
from less-selective colleges that display the requisite skills to succeed in the professional 
world. A significant increase in hiring high-ability students from less-selective colleges for 
competitive jobs would increase the motivation for additional students in these colleges to 
work harder to succeed. 


Meeting the Occam’s Razor Test 


CLA+ Career Connect is built upon the foundation of the Collegiate Learning Assessment 
(CLA+), an assessment subjected to more than a decade of critical scrutiny by members of the 
measurement science community (See Benjamin et al., 2013). The result is a test that is viewed 
as reliable and valid. The time (90 minutes) and cost (less than $50) required to take CLA+ are 
reasonable. The Career Connect system, based on education technology innovations, brings 
employers and eligible students together efficiently at little or no cost to either party. CLA+ 
Career Connect therefore meets the Occam’s razor requirement that metrics should deal with 
the most significant problem and not be multiplied unnecessarily to deal with it . 


The case to use a widely accepted critical-thinking test such as CLA+ as an additional tool to 
prescreen potential applicants to determine who employers should consider interviewing is 
compelling. Unlike suggestions that colleges and universities change their mission to focus on 
preparing students for jobs, the goal of CLA+ Career Connect is to bring many more high ability 
graduating seniors to the attention of employers. Since colleges and professors emphasize 
that critical-thinking skills are central to their mission, CLA+ Career Connect is aligned to a 
central premise that the vast majority of colleges already embrace. The critical-thinking skills 
measured by CLA+ capture a large percentage of skills desired by employers in a large number 
of fields that require a bachelor’s degree, as defined in the Bureau of Labor Statistics O*NET 
database (National Center for O*NET Development, 2014). CAE proposes using CLA+, then, as a 
pre-screening tool to reduce the “noise” between the employer and the applicants, particularly 
the graduating college seniors starting their job searches. 


TABLE 1 


@ Skill Importance 


w Skill Level 


2) CLA+ Can Be Used to Evaluate Digital Badges 


Badges are secure, verified, Internet-based credentials that include information used to 
interpret the badge outcomes. Badge data include: 
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¢ The outcomes required to earn the badge and the evidence earners provided to 
demonstrate their competence 
¢ Qualifications of the sponsor or sponsoring organization 


CLA+ can be used to evaluate the efficacy of the badge by providing evidence about the 
student’s skills as documented by a badge and CLA+ scores that measure larger skills sets 
such as critical thinking, problem solving, quantitative and qualitative reasoning and writing 
skills. The CLA+ mastery levels attained by the student provides a verification of the student’s 
badge because the CLA+ scores should be aligned with the skill levels documented by a badge. 
Here is the case for a third party role for a standardized test such as CLA+ to help employers 
interpret badges. 


Benefits of Badges 


1. College degrees and their transcripts are difficult to interpret. The same B.A. 
degree can be conferred for very different courses of study at different schools 
and sometimes for different graduating students at the same school. In a college 
transcript, information about course outcomes is not specific or transparent enough 
to help employers understand how courses map to the specific knowledge or skills 
required for career success. 

2. College grades are subjective and can be a low-quality indicator of actual competence. 
At best, they can provide a general idea of how well a student has performed. But 
there is a growing sense that grade inflation is making the college GPA unreliable 
for employers. Competency-based criteria that map learning to job requirements 
effectively complement the grades on a college transcript thus providing better 
information for employers evaluating candidates. 

3. As noted in this paper, the reputation of the institution drives the value of a degree, 
not actual work readiness. Graduates of selective colleges benefit the most 
from the brand of their college. However, every college graduate in less-selective 
colleges stands to benefit from classroom learning outcomes directly connected to 
standardized work activities and job-related skills. 

4. College transcripts and degrees do not include the skills or knowledge acquired 
outside the classroom at places such as part-time jobs, internships, or campus- 
related activities. 


Benefits for Employers, Students, and Colleges 


Employers, students, and colleges stand to benefit from the badge movement for a number 

of reasons. The information about the skill level a student obtains in a class on software or 
computer language, accounting, non-parametric statistics, or game theory can be directly 
interpreted by an employer. Students, perhaps especially in less-selective colleges, also can 
benefit significantly from the badges they achieve. They can show potential employers that they 
have the specific skills that align with career success. The colleges can also benefit from joining 
the badge movement. College instructors are already the source of many new badges based on 
the classroom learning their students are achieving. Colleges can aggregate the badges their 
instructors have developed to demonstrate how their institutions are directly impacting the 
workforce opportunities of their students. 


In short, badges are an important emerging tool that will assist in reducing the “noise” the 
market failure produces, which, in turn, is leading to an increasing mal distribution of human 
capital at the national level and greater inequality at the individual student level. 


The Problems Digital Badges Need to Overcome 


The management of identity issues has been a hurdle. As colleges and others begin to issue 
digital credentials, they need to confirm earner identity and protect the integrity of the 
credentials they issue. How can the employer judge the validity of the student’s badge? How 
can employers equate different badges that purport to measure the same skills? These and 
other issues for the burgeoning badge movement suggest a role for an appropriate standardized 
test that can be used to evaluate the efficacy of badges. 
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The Role of CLA+ CareerConnect 


CLA+ was created as a prescreening tool to increase the pool of highly qualified talent for 
employers, controlling for the positional-good bias of the current system of higher education 
that overly benefits students who go to the most selective colleges but comprise less than 10% 
of all college students. CLA+ CareerConnect gives the large number of high ability graduating 
college seniors in less-selective colleges—approximately 1.2 million each year—a greater 
opportunity to compete for the high value-added jobs their critical-thinking skill levels qualify 
them for. 


CLA+ benchmarks the minimum standards of critical thinking that colleges and employers 
both regard as requisite for college graduation and success in the workplace. Moreover, CLA+ 
CareerConnect is well suited to verify the trustworthiness and authenticity of the badges 
students put forward to employers. Each student taking CLA+ is monitored by an Internet- 
based test security organization to verify the student taking the test is the individual supposed 
to be taking the test and does not cheat. Therefore, CLA+ results can be used to corroborate 
and verify the disparate badges for which authenticity and identity may be problematic.® 


Appendix B 


See the over 80 studies of reliability and validity listed and analyzed in “The Case for Critical- 
Thinking Skills and Performance Assessment” (Benjamin et al., 2013) at cae.org. Also see Arum 
and Roksa (2014), which shows that students who did well on the CLA are in better financial 
situations and have much less unemployment five years after graduation than students who 
scored low on the CLA. 


Appendix C 


Data Sources: 

National college data were obtained from the National Center for Education Statistics, IPEDS 
website (http://nces.ed.gov/ipeds/datacenter//), and represent institutional enrollment, 
bachelor’s degrees awarded, and race/ethnicity data for all four-year degree-granting 
institutions in the database for the years 1980 and 2012. Note that enrollment data are not 
available for all institutions; these data were reported for 1,826 institutions in 1980 and 2,806 
institutions in 2012. 


CLA performance data are from all U.S. four-year institutions that tested seniors in spring 2011, 
2012, and 2013. 


Selective institutions consist of those identified by Heller (2004). Note that three of the 
institutions in Heller’s list—all part of Rutgers University—are considered a single institution 
by NCES, so the count of selective universities is smaller here (N=143 vs. N=146). Heller’s list 
is based on information provided by Barron’s selectivity college index (“Barron’s Profiles of 
American Colleges,” 2004). 


5 For more detailed information about the digital badge movement see Educause.edu (2012). Seven Things You Should Know About Badges; cf 
Pearson Vue Acclaim.com, “Unlock the Power of badges in credentialing. Acclaim-badges. Html. 
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Appendix D 


Selective Colleges and Universities: 
Amherst College, MA 

Austin College, TX 

Babson College, MA 

Barnard College, NY 

Bates College, ME 

Beloit College, WI 

Boston College, MA 

Boston University, MA 

Bowdoin College, ME 

Brandeis University, MA 

Brigham Young University-Provo, UT 
Brown University, RI 

Bryn Mawr College, PA 

Bucknell University, PA 

California Institute of Technology, CA 
Carleton College, MN 

Carnegie Mellon University, PA 

Case Western Reserve University, OH 
Claremont McKenna College, CA 
Colby College, ME 

Colgate University, NY 

College of the Atlantic, ME 

College of the Holy Cross, MA 
College of William and Mary, VA 
Colorado College, CO 

Colorado School of Mines, CO 
Columbia University in the City of New York, NY 
Connecticut College, CT 

Cooper Union for the Advancement of Science and Art, NY 
Cornell University, NY 

Dartmouth College, NH 

Davidson College, NC 

Drew University, NJ 

Duke University, NC 

Emory University, GA 

Franklin and Marshall College, PA 
Furman University, SC 

George Washington University, DC 
Georgetown University, DC 

Georgia Institute of Technology-Main Campus, GA 
Gettysburg College, PA 

Grinnell College, IA 

Grove City College, PA 

Hamilton College, NY 

Hampshire College, MA 

Harvard University, MA 

Harvey Mudd College, CA 

Haverford College, PA 

Illinois Institute of Technology, IL 
Illinois Wesleyan University, IL 
Jewish Theological Seminary of America, NY 
Johns Hopkins University, MD 
Kenyon College, OH 

Kettering University, MI 

Knox College, IL 

Lafayette College, PA 

Lawrence University, WI 

Lehigh University, PA 

Loyola University Maryland, MD 
Lyon College, AR 

Macalester College, MN 
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Massachusetts Institute of Technology, MA 
Miami University-Oxford, OH 

Middlebury College, VT 

Mount Holyoke College, MA 

New College of Florida, FL 

New York University, NY 

Northwestern University, IL 

Oberlin College, OH 

Pennsylvania State University-Main Campus, PA 
Pepperdine University, CA 

Pitzer College, CA 

Pomona College, CA 

Princeton University, NJ 

Providence College, RI 

Reed College, OR 

Rhodes College, TN 

Rice University, TX 

Rose-Hulman Institute of Technology, IN 
Rutgers University-New Brunswick, NJ 
Saint Louis University-Main Campus, MO 
Santa Clara University, CA 

Sarah Lawrence College, NY 

Scripps College, CA 

Sewanee-The University of the South, TN 
Skidmore College, NY 

Smith College, MA 

Southwestern University, TX 

St Mary’s College of Maryland, MD 

St Olaf College, MN 

Stanford University, CA 

Stevens Institute of Technology, NJ 

SUNY at Binghamton, NY 

SUNY College at Geneseo, NY 

SUNY College of Environmental Science and Forestry, NY 
Swarthmore College, PA 

Syracuse University, NY 

The College of New Jersey, NJ 

Trinity College, CT 

Trinity University, TX 

Tufts University, MA 

Tulane University of Louisiana, LA 

Union College, NY 

United States Air Force Academy, CO 
United States Coast Guard Academy, CT 
United States Merchant Marine Academy, NY 
United States Military Academy, NY 
United States Naval Academy, MD 
University of California-Berkeley, CA 
University of California-Davis, CA 
University of California-Los Angeles, CA 
University of California-Santa Barbara, CA 
University of Chicago, IL 

University of Florida, FL 

University of Georgia, GA 

University of Illinois at Urbana-Champaign, IL 
University of Mary Washington, VA 
University of Miami, FL 

University of Michigan-Ann Arbor, MI 
University of North Carolina at Chapel Hill, NC 
University of Notre Dame, IN 

University of Pennsylvania, PA 

University of Puget Sound, WA 
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University of Richmond, VA Washington University in St Louis, MO 


University of Rochester, NY Webb Institute, NY 

University of Southern California, CA Wellesley College, MA 

University of Virginia-Main Campus, VA Wesleyan College, GA 

University of Wisconsin-Madison, WI Wheaton College, IL 

Ursinus College, PA Whitman College, WA 

Vanderbilt University, TN Williams College, MA 

Vassar College, NY Worcester Polytechnic Institute, MA 
Villanova University, PA Yale University, CT 


Wake Forest University, NC 
Washington and Lee University, VA 
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